On Representing Prior Information as an Asymmetric Prior Distribution over Weights

The Bayesian modeling formalism provides a principled means of incorporating prior information into a neural network model. One can extract greater power from the Bayesian formalism through use of priors which encode particular empirical or expert knowledge. These priors will generally be asymmetric with respect to the neural network output function. It would appear that in order to compute correct expectations with repect to the posterior weight distribution that the prior have the same weight-space symmetries as the network output function (as all priors have so far). However, in this paper it is shown that the prior need not be symmetric if the likelihood function and the integrand of interest are symmetric, as they typically are. This means that constructing priors without taking neural network weight symmetries into account will indeed yield correct results with symmetric integrands; this greatly sim-pliies the construction of priors.